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DETAILED ACTION 

Information Disclosure Statement 

1 . Initialed and dated copies of Applicant's IDS form 1449, Paper No. 20041206 
and 20070504, are attached to the instant Office action. 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

3. Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. Claim 7 is drawn to functional descriptive 
material NOT claimed as residing on a computer readable medium. MPEP 
2106.IV.B.1(a) (Functional Descriptive Material) states: 

"Data structures not claimed as embodied in a computer-readable medium are 
descriptive material per se and are not statutory because they are not capable of 
causing functional change in the computer." 

"Such claimed data structures do not define any structural or functional 
interrelationships between the data structure and other claimed aspects of the invention 
which permit the data structure's functionality to be realized." 

Claim 21 , while defining a program, does not define a "computer-readable 
medium" and is thus non-statutory for that reasons. A program can range from paper 
on which the program is written, to a program simply contemplated and memorized by a 
person. The examiner suggests amending the claim to embody the program on 
"computer-readable medium" in order to make the claim statutory. 
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"In contrast, a claimed computer-readable medium encoded with the data 
structure defines structural and functional interrelationships between the data structure 
and the computer software and hardware components which permit the data structure's 
functionality to be realized, and is thus statutory." - MPEP 2106.IV.B.1(a) 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

5. Claims 1-2, 5-6, and 18-22 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Satoh et al (EP 1 1 1 7074 A2). 

Regarding claim 1 , Satoh discloses an image generating system (abstract; Figs. 
2, 4, 14, and 17), comprising: a database which stores first shape data which represents 
a three dimensional shape of a first area including at least a part of an Object area 
(paragraphs [0035] and [0063], "virtual objects"); a camera which shoots a second area 
including at least a part of the object area (Fig. 2-103, Fig. 4-103, Fig. 14-103, and Fig. 
17-103; paragraphs [0036]-[0037], [0046], and [0100], "actually sensed video") and an 
image generating apparatus which generates an image of the object area using a 
picture shot by the camera and the first shape data (Fig. 2-104, Fig. 4-104, Fig. 14-104, 
and Fig. 17-104; paragraph [0037], "AR game apparatus") , wherein said image 
generating apparatus includes: a data acquiring unit which acquires the first shape data 
from said database (Fig. 2-201, Fig. 4-201, Fig. 14-201, and Fig. 17-201; paragraph 
[0063], "game state manager"); a picture acquiring unit which acquires the picture from 
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said camera (FIG. 8-S803; paragraphs [0036], [0046], and [0071]); a first generating 
unit which generates an image of the first area by setting a predetermined viewpoint 
and a view direction and rendering the first shape data (Fig. 2-202 and 212, Fig. 4-202 
and 212, Fig. 14-202 and 212, and Fig. 17-202 and 212; paragraphs [0042], [0045], and 
[0048], "Objective viewpoint video generator" and "Subjective viewpoint video 
generator"); a second generating unit which generates an image of the second area 
when viewed from the viewpoint toward the view direction by using the picture 
(paragraphs [0044] and [0046]); and a compositing unit which composites the image of 
the first area with the image of the second area to generate the image of the object area 
(Fig. 2-203, Fig. 4-203, Fig. 14-203, and Fig. 17-203; paragraphs [0046] and [0084]; 
claim 1 "Objective viewpoint video composition unit"). 

Regarding claim 2, Satoh discloses an image generating system (abstract; Figs. 
2, 4, 14, and 17) wherein: said image generating system includes a plurality of cameras 
located at a plurality of positions (Fig. 9; paragraphs [0096]-[0102]); said image 
generating apparatus further comprises a calculating unit which calculates second 
shape data which represents a three dimensional shape of the second area using a 
plurality of the pictures acquired from said plurality of cameras (paragraphs [0056]- 
[0058]); said second generating unit sets the viewpoint and the view direction and 
renders the second shape data to generate the image of the second area (paragraphs 
[0035], [0044] and [0046]). 

Regarding claim 5, Satoh discloses an image generating system (abstract; 
Figs.2, 4, 14, and 1 7) wherein said database stores the first shape data obtained by 
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modeling an area which does not change in a short term in the object area (paragraphs 
[0007]-[0009], [0042]-[0043], [0045], and [0063]). 

Regarding claim 6, Satoh discloses an image generating system (abstract; 
Figs.2, 4, 14, and 17) wherein said database stores the first shape data obtained by 
modeling an area which does not change in a short term in the object area (paragraphs 
[0007]-[0009], [0042]-[0043], [0045], and [0063]). 

Regarding claim 18, Satoh discloses an image generating apparatus (abstract; 
Figs. 2, 4, 14, and 17), comprising: a data acquiring unit which acquires first shape data 
which represents a three dimensional shape of a first area including at least one part of 
an object area from a database (Fig. 2-201, Fig. 4-201, Fig. 14-201, and Fig. 17-201; 
paragraph [0063], "game state manager") which stores the first shape data (paragraphs 
[0035] and [0063], "virtual objects"); a picture acquiring unit which acquires a picture of 
a second area including at least one part of the object area shot (Fig. 2-103, Fig. 4-103, 
Fig. 14-103, and Fig. 17-103; paragraphs [0036]-[0037], [0046], and [0100], "actually 
sensed video") by a plurality of cameras located at a plurality of positions from the 
cameras (Fig. 9; paragraphs [0096]-[0102]); a first generating unit which generates an 
image of the first area by setting a predetermined viewpoint and a view direction and 
rendering the first shape data (Fig. 2-202 and 212, Fig. 4-202 and 212, Fig. 14-202 and 
212, and Fig. 17-202 and 212; paragraphs [0042], [0045], and [0048], "Objective 
viewpoint video generator" and "Subjective viewpoint video generator"); a second 
generating unit which generates an image of the second area when viewed from the 
viewpoint toward the view direction by using the picture shot (paragraphs [0044] and 
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[0046]); and a compositing unit which composites the image of the first area with the 
image of the second area to generate the image of the object area (Fig. 2-203, Fig. 4- 
203, Fig. 14-203, and Fig. 17-203; paragraphs [0046] and [0084]; claim 1 "Objective 
viewpoint video composition unit"). 

Regarding claim 19, claim 19 is analogous and corresponds to claim 18. See 
rejection of claim 18 for further explanation. 

Regarding claim 20, Satoh discloses an image generating method (abstract; 
Figs. 2, 4, 14, and 17), wherein when generating an image of an object area viewed 
from a predetermined viewpoint toward a predetermined view direction (Fig. 2-103, Fig. 
4-103, Fig. 14-103, and Fig. 17-103; paragraphs [0036]-[0037], [0044], [0046], and 
[0100], "actually sensed video") using a plurality of pictures shot by a plurality of 
cameras (Fig. 9; paragraphs [0096]-[0102]); and acquired from the cameras in real time 
(paragraphs [0088]-[0089]), the method generating the image of the object area which 
represents a present state of the object area artificially by complementing the pictures 
with an image (Fig. 2-203, Fig. 4-203, Fig. 14-203, and Fig. 17-203; paragraphs [0046] 
and [0084]; claim 1 "Objective viewpoint video composition unit") generated by using 
three-dimensional shape data obtained by modeling at least a part of the object area 
(Fig. 2-201, Fig. 4-201, Fig. 14-201, and Fig. 17-201; paragraphs [0035] and [0063], 
"game state manager" and "virtual object"). 

Regarding claim 21, Satoh discloses a program executable by a computer 
(paragraphs [0061]-[0062]; claims 19-27). Moreover, claim 21 is analogous and 
corresponds to claim 18. See rejection of claim 18 for further explanation. 
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Regarding claim 22, Satoh discloses a computer-readable recording medium 
which stores a program executable by a computer (paragraphs [0061]-[0063]; claims 
19-27). Moreover, claim 22 is analogous and corresponds to claim 18. See rejection of 
claim 1 8 for further explanation. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 3 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Satoh et al (EP 1 1 1 7074 A2) in view of Yonezawa et al. (US 2002/0075286). 

Regarding claim 3, Satoh discloses all the previous claim limitations except the 
claim limitation of claim 3. However, Yonezawa discloses compositing unit generates 
the image of the object area by complementing an area that is not represented by the 
second shape data with the image of the first area generated from the first shape data 
(paragraphs [0046]-[0047] and [0049]-[0050]). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Yonezawa's invention in Satoh's invention to 
provide a more real composite image as suggested by Yonezawa (paragraph [0006]). 

Regarding claim 7, Satoh further discloses an image generating system 
(abstract; Figs. 2, 4, 14, and 17) wherein said database stores the first shape data 
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obtained by modeling an area which does not change in a short term in the object area 
(paragraphs [0007]-[0009], [0042]-[0043], [0045], and [0063]). 

8. Claims 4 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Satoh et al (EP 1 1 1 7074 A2) in view of Anabuki et al. (US 6,633,304). 

Regarding claim 4, Satoh discloses all the previous claim limitations except the 
claim limitation of claim 4. However, Anabuki discloses second generating unit renders 
the area which is not represented by the second shape data with a transparent color 
when rendering the second shape data (col.7, lines 19-25); said compositing unit 
generates the image of the object area by overwriting the image of the second area with 
the image of the first area (col. 2, lines 16-24). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Anabuki's invention in Satoh's invention to provide 
a mixed reality presentation apparatus and control method suggested by Anabuki (col. 
2, lines 16-20). 

Regarding claim 8, Satoh further discloses an image generating system 
(abstract; Figs.2, 4, 14, and 17) wherein said database stores the first shape data 
obtained by modeling an area which does not change in a short term in the object area 
(paragraphs [0007]-[0009], [0042]-[0043], [0045], and [0063]). 

9. Claims 9-1 0, and 1 3-1 5 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Satoh et al (EP 1 117074 A2) in view of Ikeuchi ("Modeling from 
Reality"). 
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Regarding claim 9, Satoh discloses all the previous claim limitations except the 
claim limitation of claim 9. However, Ikeuchi teaches a database stores first color data 
which represents a color of the first area (section 3); said image generating apparatus 
further includes a lighting calculating unit which calculates a situation of a lighting in the 
picture shot by comparing the first color data acquired from said database with color 
data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Ikeuchi's method in Satoh's invention to effectively 
use for synthesizing realistic object images as suggested by Ikeuchi (section 6). 

Regarding claim 10, Ikeuchi further teaches a database stores first color data 
which represents a color of the first area (section 3); said image generating apparatus 
further includes a lighting calculating unit which calculates a situation of a lighting in the 
picture shot by comparing the first color data acquired from said database with color 
data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

Regarding claim 12, Ikeuchi further teaches a database stores first color data 
which represents a color of the first area (section 3); said image generating apparatus 
further includes a lighting calculating unit which calculates a situation of a lighting in the 
picture shot by comparing the first color data acquired from said database with color 
data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

Regarding claim 13, Ikeuchi further teaches a database stores first color data 
which represents a color of the first area (section 3); said image generating apparatus 
further includes a lighting calculating unit which calculates a situation of a lighting in the 
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picture shot by comparing the first color data acquired from said database with color 
data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

Regarding claim 14, Ikeuchi further teaches first generating unit adds an effect of 
lighting similar to the lighting in the picture shot to the image of the first area in 
consideration of the situation of the lighting (sections 3.1, 3.2, 4, 4.1, 4.2, and 4.3). 

Regarding claim 15, Ikeuchi further teaches first generating unit adds a 
predetermined effect of lighting to the image of the first area (sections 3.1 , 3.2, 4, 4.1 , 
4.2, and 4.3); said second generating unit adds the predetermined effect of lighting to 
the image of the second area, after once removing the effect of 
lighting from the image of the second area (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

10. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Satoh et 
al (EP 1 1 17074 A2) in view of Yonezawa et al. (US 2002/0075286), and further in view 
of Ikeuchi ("Modeling from Reality"). 

Regarding claim 1 1 , Satoh and Yonezawa disclose all the previous claim 
limitations except the claim limitation of claim 1 1 . However, Ikeuchi teaches a database 
stores first color data which represents a color of the first area (section 3); said image 
generating apparatus further includes a lighting calculating unit which calculates a 
situation of a lighting in the picture shot by comparing the first color data acquired from 
said database with color data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Yonezawa's invention and Ikeuchi's method in 
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Satoh's invention to synthesizing realistic object images as suggested by Ikeuchi (page 
124; section 6). 

1 1 . Claims 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Satoh 
et al (EP 1 1 1 7074 A2) in view of Anabuki et al. (US 6,633,304), and further in view of 
Ikeuchi ("Modeling from Reality"). 

Regarding claim 9, Satoh and Anabuki disclose all the previous claim limitations 
except the claim limitation of claim 9. However, Ikeuchi teaches a database stores first 
color data which represents a color of the first area (section 3); said image generating 
apparatus further includes a lighting calculating unit which calculates a situation of a 
lighting in the picture shot by comparing the first color data acquired from said database 
with color data of the picture shot (sections 3.1 , 3.2, 4, 4.1 , 4.2, and 4.3). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Anabuki's invention and Ikeuchi's method in Satoh's 
invention to effectively use for synthesizing realistic object images as suggested by 
Ikeuchi (section 6). 

12. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Satoh et 
al (EP 1 1 17074 A2) in view of Kondo (US 6,812,924), and further in view of Sawada 
(US 5,844,625). 

Regarding claim 16, Satoh discloses all the previous claim limitations, including 
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the image generating system (abstract; Figs. 2, 4, 14, and 17), a database storing a 
plurality of the first shape data corresponding to the object areas of a plurality of times 
(paragraph [0063]), and the image generating apparatus (Fig. 2-104, Fig. 4-104, Fig. 
14-104, and Fig. 17-104; paragraph [0037], "AR game apparatus"), except the other 
claim limitations of claim 16. However, Kondo discloses a first selecting unit which 
selects the first shape data to be acquired by the data acquiring unit among the plurality 
of the first shape data stored in said database (Figs. 24 and 25; dol. 10, lines 18-26; col. 
11, lines 20-29; claim 7). Sawada discloses storing the picture shot(col. 2, lines 43-49, 
col. 5, lines 28-37) and a second selecting unit which selects the picture shot to be 
acquired by the picture acquiring unit among the pictures stored in said recording 
apparatus (col. 5, lines 64-67; col. 6, lines 1-6). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Kondo's invention and Sawada's invention in 
Satoh's invention to provide a an apparatus for obtaining shape data (Kondo, col. 1, 
lines 48-50) and a picture processing apparatus (Sawada, col. 2, lines 20-21). 

13. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Satoh et 
al (EP 1 1 1 7074 A2) in view of Kondo (US 6,812,924), and further in view of Sawada 
(US 5,844,625) and Yonezawa et al. (US 2002/0075286). 

Regarding claim 17, Satoh, Kondo, and Sawada disclose all the previous claim 
limitations except the claim limitation of claim 17. However, Yonezawa discloses 
second selecting unit selects the first shape data corresponding to the time when the 



Application/Control Number: 10/780,303 Page 13 

Art Unit: 2624 

picture selected by said first selecting unit was shot (paragraphs [0098]-[0099], 
"synchronization"). 

It would have been prima facie obvious to one of ordinary skill in the art at the 
time the invention was made to use Kondo's invention, Sawada's invention, and 
Yonezawa's invention in Satoh's invention to provide a more real composite image as 
suggested by Yonezawa (paragraph [0006]). 

Conclusion 

r 14. No claims are allowed. 
1 5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to John Wahnkyo Lee whose telephone number is (571 ) 
272-9554. The examiner can normally be reached on Monday - Friday (Alt.) 7:30 a.m. - 
5:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
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Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




